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PREFACE 


This  work  was  performed  under  the  “Advanced  Distributed  Learning  Common 
Framework”  task.  Technical  cognizance  for  this  task  is  assigned  to  Dr.  Robert  Wisher, 
Office  of  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness,  Readiness  and 
Training  Directorate. 
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EXECUTIVE  SUMMARY 


The  Advanced  Distributed  Learning  (ADL)  initiative  was  undertaken  by  the 
Department  of  Defense  (DoD)  at  the  request  of  the  White  House  Office  of  Science  and 
Technology  Policy  (OSTP)  and  in  cooperation  with  the  other  Federal  Agencies.  Its  goal 
is  to  make  education,  training,  and  performance  aiding  accessible  anytime  and  anywhere. 
This  goal  is  being  pursued  through  intense  and  frequent  interactions  among  industry, 
government,  and  academic  participants.  ADL  specifications  are  now  being  adopted 
across  Europe,  Asia,  the  Pacific  Rim,  and  the  Americas. 

In  the  near  future,  much  of  what  we  learn  may  be  based  on  open,  anytime,  any¬ 
where  interactions  accessing  something  that  approximates  the  whole  of  human  knowl¬ 
edge.  This  access  will  be  achieved  through  a  variety  of  capabilities,  many  of  which  (e.g., 
computer  technology,  global  communications,  and  the  World  Wide  Web  or  its  succes¬ 
sors)  are  already  in  place.  Interactions  based  on  this  access  can  be  tailored  to  each 
learner’s  abilities,  prior  knowledge,  interests,  and  preferred  learning  style(s).  The  ADL 
vision  assumes  that  interactions  will  be  generated  in  real  time  and  on-demand  to  allow 
the  free-flowing,  mixed  initiative  give-and-take  that  is  characteristic  of  true  conversation. 

These  education,  training,  and  performance-aiding  interactions  may  take  place  as 
conversations  that  address  what  the  user  needs  or  wants  to  know  at  the  moment.  They  are 
likely  to  be  mediated  by  personal  learning  associates  that  will  be  wireless  and  voice  inter¬ 
active  but  also  capable  of  most  other  modes  of  interaction,  including  a  full  range  of 
media.  Physically,  these  devices  may  be  hand  held  (possibly  combined  with  the  game¬ 
playing,  photograph-taking,  instant-messaging,  video  telephones  we  have  today),  worn, 
or  even  implanted. 

If  real-time,  on-demand  interactions — conversational  or  not — are  to  tap  into  the 
World  Wide  Web,  they  will  need  to  identify  and  access  objects  that  are  accessible,  port¬ 
able,  durable,  and  reusable.  Development  of  the  Sharable  Content  Object  Reference 
Model  (SCORM)  was  based  on  these  considerations.  SCORM  defines  the  interrelation¬ 
ship  of  objects,  data  models,  and  protocols  so  that  instructional  objects  can  be  shared 
across  all  confonning  systems.  SCORM  is  now  stabilized  and  being  used  to  create 


ES-1 


education,  training,  and  performance-aiding  materials.  More  than  5  million  objects  con¬ 
forming  to  SCORM  specifications  exist  and  are  in  use. 

The  Content  Object  Repository,  Discovery,  and  Resolution  Architecture 
(CORDRA)  is  a  related  development  being  undertaken  in  cooperation  with  the  Corpora¬ 
tion  for  National  Research  Initiatives  (CNRI).  It  is  focused  on  identifying  (“discovering”) 
and  then  locating  the  precise  object(s)  needed  for  an  application  or  an  interaction. 
CORDRA  will  rely  on  metadata  to  specify  and  locate  objects,  but  its  system  of  registries 
will  allow  access  to  any  registered,  but  locally  controlled,  repository.  Its  naming  conven¬ 
tions  will  track  objects  as  they  are  modified  and  moved  from  repository  to  repository. 

The  Semantic  Web,  which  is  being  developed  under  the  auspices  of  the  World 
Wide  Web  Consortium,  is  also  included  in  ADL  planning.  It  is  intended  to  improve  coop¬ 
eration  between  computers  and  human  beings  by  imbuing  Web  information  with 
meaning.  Through  a  system  of  ontologies,  it  will  identify  and  expose  semantic  linkages 
between  bodies  of  knowledge  regardless  of  how  disparate  they  appear  to  be.  It  will  allow 
more  comprehensive  and  substantive  models  of  subject  matter  domains  and  learners’  lev¬ 
els  of  mastery  than  we  now  have  and  combine  them  with  more  precise  discovery  of  the 
instructional  objects  needed  to  produce  desired  human  competencies. 

Four  possibilities  (among  others)  in  education  and  training  may  arise  from  the 
ADL  environment: 

1 .  Less  predefined  sequencing.  Instructional  design  as  a  process  of  prespeci¬ 
fying  and  predefining  a  sequence  of  activities  must  evolve  if  the  ADL  vision 
of  on-demand,  real-time  access  is  to  be  realized  fully. 

2.  More  assessment  and  fewer  tests.  Assessment  will  become  continuous  and 
unobtrusive  as  capabilities  evolve  for  developing  a  model  of  the  learner  from 
routine  interactions. 

3.  Fewer  “lessons.”  The  notion  of  instructional  modules  will  need  to  evolve  to 
support  instructional  interactions  generated  on-demand  and  in  real  time. 

4.  More  reliable  outcomes.  Instruction  will  need  to  become  more  of  an  engi¬ 
neering  discipline  in  which  specific  outcomes  are  reliably  achieved  for  all 
learners.  This  discipline  will  rely  on  detailed  knowledge  of  the  learner’s  prior 
and  developing  competence  matched  with  comprehensive  representations  of 
the  subject  matter  and  instructional  techniques. 

With  or  without  ADL,  fully  accessible,  anytime,  anywhere  education,  training, 
and  performance  aiding  seem  inevitable.  They  are  likely  to  require  structural  and  organ¬ 
izational  changes  in  current  procedures  and  practices.  Sooner  or  later,  existing 
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organizations  must  deal  with  this  evolution  and  its  emerging  processes,  capabilities,  and 
opportunities.  It  may  be  time  to  begin  addressing  the  procedural  and  administrative 
opportunities  and  issues  presented  by  ADL  technical  capabilities. 


ES-3 


THE  ADVANCED  DISTRIBUTED  LEARNING  (ADL)  VISION  AND 
GETTING  FROM  HERE  TO  THERE 


A.  ABOUT  ADL 

The  ADL  initiative  was  undertaken  by  the  Department  of  Defense  (DoD)  at  the 
request  of  the  White  House  Office  of  Science  and  Technology  Policy  (OSTP)  and  in 
cooperation  with  the  other  Federal  agencies.  It  is  intended  to  provide  a  model  for  all  Fed¬ 
eral  agencies  in  making  education,  training,  and  perfonnance  aiding  readily  accessible — 
anytime,  anywhere.  It  is  being  developed  through  intense  and  frequent  collaboration 
among  industry,  government,  and  academic  participants,  and  its  specifications  are  being 
adopted  across  Europe,  Asia,  the  Pacific  Rim,  and  the  Americas. 

ADL  is  expected  to  deliver  both  training  and  education.  Training  is  a  means  to  an 
end  and  prepares  individuals  to  perfonn  specific  tasks  and  jobs.  Education  is  an  end  in 
itself  and  prepares  individuals  for  life.  The  knowledge  representations  and  user  interac¬ 
tions  underlying  these  instructional  applications  are  identical  to  those  needed  to  assist 
users  in  decision-making,  planning,  problem  solving,  maintaining  and  operating  equip¬ 
ment,  and  other  perfonnance-aiding  functions.  For  these  reasons,  along  with  the  costs  to 
be  saved  by  providing  such  functions  (e.g.,  Fletcher  and  Johnston,  2002),  perfonnance 
aiding  and  instruction  are  significant  objectives  of  the  ADL  initiative  (Wisher  and 
Fletcher,  2004;  Dodds  and  Fletcher,  2004).  However,  perfonnance  aiding  is  a  substantial 
topic  by  itself,  and,  in  this  document,  space  considerations  limit  the  discussion  of  its  pos¬ 
sibilities,  promise,  and  implications. 

The  implications  of  the  ADL  initiative  are  not  limited  to  Federal  agencies.  It 
presents  opportunities  and  challenges  to  many  other  settings,  including  classroom 
instruction.  Its  anytime,  anywhere  instructional  goals  include  classrooms,  workplaces, 
conference  rooms,  job  sites,  and  homes.  However,  the  wide  access  to  instruction  that 
ADL  will  provide  students  outside  traditional  instructional  institutions  and  venues 
challenges  instructors,  administrators,  policy  makers,  and  students  themselves  to  assume 
new  and  unaccustomed  roles  with  new  and  unaccustomed  responsibilities. 
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With  or  without  ADL,  fully  accessible  anytime,  anywhere  education  (and  training 
and  performance  aiding)  is  likely  to  become  a  reality.  If  nothing  else,  ADL  is  a  harbinger 
of  future  learning  processes,  capabilities,  and  opportunities  that  our  instructional  institu¬ 
tions  must  address  sooner  or  later.  If  they  do  so  successfully,  their  efforts  will  benefit 
everyone  concerned  with  human  learning  and  performance. 

B.  THE  ADL  VISION 

The  ADL  initiative  is  based  on  the  view  of  future  education,  training,  and  per¬ 
formance  aiding  illustrated  in  Figure  1.  As  the  figure  suggests,  this  view,  or  “vision,” 
keys  on  three  main  components: 

1.  A  Global  Information  Grid — currently  the  World  Wide  Web — shown  as 
the  cloud  on  the  left  side  of  the  figure  provides  an  infrastructure  populated  by 
reusable  instructional  objects 

2.  A  Server,  shown  in  the  middle  of  the  figure  locates  and  then  assembles 
instructional  objects  into  education,  training,  and/or  perfonnance-aiding 
materials  tailored  to  user  needs 

3.  Devices,  shown  on  the  right  side  of  the  figure,  can  be  carried  or  worn  and 
serve  as  personal  learning  associates  that  deliver  education,  training,  and  per¬ 
formance  aiding  to  users  anywhere,  anytime. 

From  an  instructional  point  of  view,  the  critical  element  in  this  vision  is  the 
Server.  It  will  assemble  material  needed  to  support  interactions  with  learners  and  users 
on-demand  and  in  real  time.  These  interactions  will  be  tailored  to  the  needs,  capabilities, 
intentions,  and  learning  state  of  each  individual  or  group  of  individuals.  ADL’s  long-term 
vision  is  to  establish  instruction  and  performance-aiding  interactions  between  technology 
and  (human)  users  that  consist  of  goal-directed  conversations  tailored  to  each  learner’s  or 
user’s  needs,  skills,  knowledge,  abilities,  and  interests. 

Such  goal-directed  conversations  must  draw  on  effective  instructional  strategies, 
accurate  representations  of  the  user,  and  comprehensive  representations  of  relevant  sub¬ 
ject  matter.  Generative  systems  of  this  sort  have  been  the  objective  of  DoD  research  and 
development  (R&D)  investment  since  the  mid-1960s  (Carbonell,  1970;  Fletcher  and 
Rockway,  1986;  Fletcher,  1988).  They  are  the  original  goal  of  what  today  are  called 
Intelligent  Tutoring  Systems  (ITSs).  DoD  supported  generative  capabilities  not  so  much 
as  a  clever  application  of  artificial  intelligence  (AI),  but  as  a  way  to  reduce  instructional 
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Sharable  content  objects  (SCOs)  from  across  the  World  Wide  Web  assembled  in 
time,  on-demand  to  provide  learning  and  assistance  anytime,  anywhere. 

Figure  1.  An  ADL  Future 

materials’  preparation  by  substituting  the  capital  of  technology  and  automation  for 
human  labor  (Fletcher  and  Rockway,  1986).  These  capabilities  remain  essential  to 
achieving  the  long-term  goals  of  ADL. 

C.  THE  BASIS  FOR  ADL 

ADL  goals  arise  from  four  main  technological  opportunities: 

1 .  Advances  in  electronics 

2.  The  pervasive  accessibility  of  the  World  Wide  Web 

3.  Capabilities  developed  for  ITSs 

4.  Emerging  specifications  for  reusable,  sharable  instructional  objects. 

1.  Electronics 

The  first  and  most  obvious  technological  opportunity  keys  on  the  rapid  develop¬ 
ment  of  digital  electronics  spurred  on  by  the  operation  of  Moore’s  Law  (Service,  1996; 
Mann,  2000).  As  readers  may  recall,  Gordon  Moore  is  a  semiconductor  pioneer  and 
co-founder  of  the  Intel  Corporation.  In  1965,  Moore  noted  that  engineers  were  doubling 
the  number  of  electronic  devices  (basically  transistors)  on  chips  every  year.  In  1975,  he 
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revised  his  prediction  to  say  that  the  doubling  would  occur  every  2  years.  If  we  split  the 
difference  and  predict  that  it  will  occur  every  18  months,  our  expectations  fit  reality  quite 
closely. 

A  consequence  of  Moore’s  Law  is  that  computers  initially  selling  for  $3,000  may 
cost  about  half  that  amount  18  months  later.  Another  consequence  is  that  the  delivery 
devices  shown  on  the  right  side  of  Figure  1  will  continue  to  decrease  as  much  in  physical 
size  and  cost  as  they  will  increase  in  functionality — not  unlike  today’s  cellular  tele¬ 
phones,  which  may  provide  ubiquitous  platfonns  for  ADL  delivery. 

2.  The  World  Wide  Web 

The  second  technological  opportunity  underlying  ADL  arises  from  the  develop¬ 
ment  and  implementation  of  the  Global  Information  Grid,  which  currently  takes  the  form 
of  the  World  Wide  Web.  The  Web  made  feasible  DoD’s  goal  of  accessible  education, 
training,  and  performance  aiding  available  anytime  and  anywhere.  Development  of  the 
Semantic  Web  will  contribute  substantially  to  achieving  the  long-tenn  interactive 
instructional  goals  of  ADL  by  providing  the  semantic  linkages  needed  to  create  compre¬ 
hensive  representations  of  subject  matter,  expertise,  learners,  and  users. 

3.  ITSs 

The  third  technological  opportunity  is  presented  by  the  generative  capabilities  of 
ITSs.  Successful  ADL  development  assumes  that  its  functionalities  must  be  tailored  to 
the  specific  needs,  abilities,  goals,  and  interests  of  the  individual  student  or  user 
(Fletcher,  2003;  Wisher  and  Fletcher,  2004;  Dodds  and  Fletcher,  2004).  This  tailoring,  or 
individualization,  is  as  critical  for  performance  aiding  as  it  is  for  education  and  training. 
ADL  functions  are  expected  to  provide  what  Brown,  Burton,  and  DeKleer  (1982)  called 
articulate  expertise.  Not  only  must  these  functions  provide  helpful  and  relevant  guidance, 
but  they  must  do  so  in  a  way  that  the  learners  and  users,  who  have  varying  levels  of 
knowledge  and  skill,  can  understand.  ADL  will,  therefore,  need  to  be  “intelligent” — 
building  on  capabilities  that  have  been  the  developmental  province  of  ITS. 

At  this  point,  emphasizing  the  capabilities  provided  by  “non-intelligent,” 
computer-based  instruction  programs  since  the  1950s  may  be  worthwhile.  These  pro¬ 
grams  have  been  able  to 

•  Accommodate  an  individual  student’s  rate  of  progress,  allowing  as  much  or 
as  little  time  needed  by  each  student  to  reach  instructional  objectives 
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•  Tailor  the  content  and  the  sequence  of  instructional  content  to  each  student’s 
needs 

•  Make  the  instruction  as  easy  or  difficult,  specific  or  abstract,  applied  or  theo¬ 
retical  as  necessary 

•  Adjust  to  a  student’s  most  efficient  learning  styles  (collaborative  or  individ¬ 
ual,  verbal  or  visual,  and  so  forth). 

These  capabilities  have  been  described,  discussed,  and  reviewed  by  Galanter 
(1959),  Atkinson  and  Wilson  (1969),  Suppes  and  Morningstar  (1972),  Fletcher  and 
Rockway  (1986),  and  many  others.  To  one  degree  or  another,  they  have  been  imple¬ 
mented  and  available  in  computer-based  instruction  (CBI)  since  its  inception. 

“Intelligence”  in  ITSs  is  a  different  matter  and  more  than  a  marketing  tenn.  When 
it  was  first  introduced  into  CBI,  it  concerned  specific  goals.  The  distinction  between  ITSs 
and  other  CBI  was  keyed  to  these  goals.  The  phrase  “intelligent  tutoring  system”  implied 
serious  and  clear  technological  advances.  It  referred  to  specific  capabilities  that  were  first 
targeted  in  the  1960s  (Carbonell,  1970;  Sleeman  and  Brown,  1982).  Two  of  these 
defining  capabilities  are  that  ITSs  (1)  allow  either  the  system  or  the  student  to  ask  open- 
ended  questions  and  initiate  instructional,  “mixed-initiative”  dialogue  as  needed  or 
desired  and  (2)  generate  instructional  material  and  interactions  on-demand  rather  than 
requiring  developers  to  foresee  and  pre-store  all  the  materials  and  interactions  needed  to 
meet  all  possible  eventualities. 

Mixed-initiative  dialogue  requires  a  language  for  infonnation  retrieval,  decision- 
aiding,  and  instruction  that  is  shared  by  the  system  and  the  student/user.  Natural  language 
has  been  a  frequent  choice  for  this  capability  (e.g.,  Brown,  Burton,  and  DeKleer,  1982; 
Graesser,  Person,  and  Magliano,  1995),  but  the  language  of  mathematics,  mathematical 
logic,  electronics,  and  other  well-structured  communication  systems  have  also  been  used 
(Barr,  Beard,  and  Atkinson,  1975;  Suppes,  1981;  Sleeman  and  Brown,  1982;  Psotka, 
Massey,  and  Mutter,  1988;  Woolf  and  Regian,  2000). 

Generative  capability  requires  the  system  to  devise  interactions  with  students  on- 
demand — not  draw  from  predicted  and  pre-stored  formats  This  capability  involves  not 
only  generating  problems  tailored  to  each  student’s  needs,  but  also  providing  coaching, 
hints,  critiques  of  completed  solutions,  appropriate  and  effective  teaching  strategies,  and, 
overall,  the  interactions  and  presentations  needed  for  one-on-one  tutorial  instruction. 
These  interactions  must  be  generated  from  information  primitives  using  an  “instructional 
grammar”  that  is  analogous  to  the  deep  structure  grammar  of  linguistics. 
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Motivations  for  the  aforementioned  capabilities  can  be  found  in  the  perennial 
desire  for  cost  containment  accomplished  by  generating  rather  than  anticipating  and  pre¬ 
storing  responses  to  all  possible  student  states  and  actions.  However,  they  also  arise  from 
basic  research  on  human  learning,  memory,  perception,  and  cognition.  In  the  1960s,  as 
documented  by  Neisser  (1967)  among  others,  the  emphasis  in  basic  research  and  under¬ 
standing  of  human  behavior  shifted  from  the  strict  logical  positivism  of  behavioral  psy¬ 
chology,  which  focused  on  directly  observable  actions,  to  consideration  of  the  internal, 
cognitive  processes  that  were  needed  to  explain  empirically  observed  behavioral 
phenomena  and  are  assumed  to  mediate  and  enable  human  learning. 

The  hallmark  of  this  approach  is  the  understanding  that  seeing,  hearing,  and 
remembering  are  all  acts  of  construction,  making  more  or  less  use  of  the  limited  stimulus 
information  provided  by  our  perceptual  capabilities.  Constructivist  approaches  are  the 
subject  of  much  current  and  relevant  discussion  in  instructional  research  circles  (e.g., 
Duffy  and  Jonanssen,  1992;  Tobias  and  Frase,  2000),  but  they  are  firmly  grounded  in  the 
primordial  foundations  of  scientific  psychology.  For  instance,  William  James 
(1890/1950)  stated  his  General  Law  of  Perception  as  follows:  “Whilst  part  of  what  we 
perceive  comes  through  our  senses  from  the  object  before  us,  another  part  (and  it  may  be 
the  larger  part)  always  comes  out  of  our  mind”  (p.  747). 

In  this  sense,  the  generative  capability  sought  by  ADL  and  ITS  developers  is  not 
merely  something  nice  to  have  but  is  essential  if  we  are  to  advance  beyond  the 
constraints  of  the  prescribed,  pre-branched,  programmed  learning,  and  ad-hoc  principles 
commonly  used  to  design  CBI.  We  need  an  interactive,  generative  capability  if  we  are  to 
deal  successfully  with  the  extent,  variety,  and  mutability  of  human  cognition. 

As  stated,  the  ADL  vision  is  that  training,  education,  and  performance  aiding  will 
take  the  form  of  human-computer  conversations.  This  capability  has  been  realized  in 
systems  that  could  converse  in  a  fonnal  language,  such  as  computer  programming  [e.g., 
Binary  Integer  Programming  (BIP)  (Barr,  Beard,  and  Atkinson  (1975)]  or  propositional 
calculus  [e.g.,  EXCHECK  (Suppes,  1981)]  or  could  base  the  conversation  on  determinate 
technical  phenomena  using  clearly  defined  and  well  understood  terms  [e.g.,  SOPHIE, 
(Brown,  Burton,  and  DeKleer,  1982)].  More  recent  research,  such  as  that  presented  by 
Graesser,  Gernsbacher,  and  Goldman  (2003),  suggests  that  significant  natural  language 
dialogue  capabilities  are  achievable.  Currently,  however,  they  remain  closer  to  the 
“bleeding  edge”  of  technology  than  to  the  mainstream.  Although  the  ability  to  assemble 
reusable  objects  in  real  time  and  on-demand  into  meaningful  instructional  or 
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performance-aiding  conversations  has  yet  to  be  demonstrated  conclusively,  progress  in 
the  development  of  ITSs  suggests  that  it  is  not  an  unreasonable  goal. 

4.  Instructional  Objects 

Finally,  ADL  keys  on  technological  opportunities  offered  by  instructional  objects. 
ADL  development  is  presently  focused  on  packaging  instructional  objects  in  anticipation 
of  what  has  been  called  by  Spohrer,  Sumner,  and  Shum  (1998)  the  “educational  object 
economy.”  In  such  an  economy,  the  emphasis  in  preparing  materials  for  technology- 
based  instruction  (or  performance  aiding)  will  shift  from  the  current  concern  with 
developing  instructional  objects  themselves  to  one  of  integrating  already  available 
objects  into  meaningful,  relevant,  and  effective  interactions. 

The  recent  evolution  of  instructional  objects  has  not  escaped  the  attention  of 
researchers.  In  assessing  the  educational  value  of  objects,  Roschelle  and  Kaput  (1995) 
emphasized  the  ability  to  combine  many  kinds  of  interactive  content  in  multiple  display 
formats  and  obtain  for  education  the  benefits  now  being  realized  in  business  from  the  use 
of  integrated  office  software.  Roschelle  et  al.  (1999)  described  software  technologies 
underlying  the  development  of  five  object-based  education  projects  and  reviewed  their 
relative  effectiveness.  Gibbons,  Nelson,  and  Richards  (2000)  reviewed  in  detail  the 
nature  and  value  of  instructional  objects  for  educational  applications  and  concluded, 
along  with  Wiley  (2000),  that  instructional  objects  are  the  technology  of  choice  in 
supporting  the  evolution  of  technology-based  instruction  because  of  their  potential  for 
reusability,  adaptability,  and  scalability.  Wiley  (2000)  provided  the  first  scholarly,  book- 
length  treatment  of  instructional  objects,  and  others  may  be  expected  to  follow.  However, 
beyond  the  economies,  capabilities,  and  other  benefits  discussed  by  these  researchers  is 
the  possibility  that  the  development  of  sharable  instructional  objects  opens  the  door  to 
genuinely  generative  instruction. 

Instructional  objects  may  then  supply  the  primitives  from  which  instructional 
interactions  can  be  created  on-demand  and  in  real  time  and  serve  as  reusable  instructional 
components  that  reduce  the  costs  of  developing  basic  instructional  materials,  including 
graphics,  simulations,  and  simulation  scenarios  (e.g.,  Towne,  1998,  2003).  An  economi¬ 
cally  viable,  generative  capability  for  instruction,  in  turn,  depends  on  specifications  for 
the  development  of  sharable  instructional  objects. 

Developing  these  specifications  has  become  a  significant  activity  of  the  ADL  Ini¬ 
tiative.  The  specifications  are  intended  to  separate  objects  from  context-specific,  run-time 
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constraints  and  proprietary  systems  so  that  the  objects  can  be  incorporated  into  other 
applications.  They  prescribe  common  interfaces  and  data  interchange  procedures  so 
instructional  objects  can  be  aggregated  into  assemblies  that  guide  and  assist  learners  and 
users.  Instructional  objects  so  specified  must  be 

•  Accessible.  Objects  in  one  remote  location  should  be  able  to  be  identified, 
located,  and  accessed  and  then  delivered  to  many  other  locations  when  and 
where  they  are  needed. 

•  Interoperable.  Objects  developed  in  one  location  with  one  set  of  tools  or 
platform  should  be  accessible  and  usable  in  other  locations  with  different 
tools  operating  in  different  environments  on  different  platforms. 

•  Durable.  Objects  should  be  able  to  withstand  technology  changes  (including 
version  changes  and  upgrades)  without  having  to  be  redesigned,  reconfig¬ 
ured,  or  recoded. 

•  Reusable.  Objects  should  be  sufficiently  flexible  to  be  used  in  multiple 
applications  and  contexts. 

These  criteria  underlie  the  Sharable  Content  Object  Reference  Model  (SCORM), 
which  is  an  evolving  specification  for  creating  instructional  objects  that  meet  these  crite¬ 
ria  (Dodds,  2002;  Wisher  and  Fletcher,  2004;  Dodds  and  Fletcher,  2004).  The  SCORM 
specification  is  by  no  means  the  totality  of  ADL,  but  it  provides  a  basis  for  populating  the 
World  Wide  Web,  or  whatever  fonn  the  Global  Information  Grid  takes  in  the  future,  with 
a  ready  supply  of  accessible,  useable  instructional  objects.  Once  these  objects  exist,  they 
must  be  identified,  assessed,  selected,  and  assembled  in  real  time  and  then  handed  to 
devices  that  deliver  instruction  and/or  perfonnance  aiding.  As  stated,  this  is  the  job  of  the 
Server  shown  in  the  middle  of  Figure  1.  By  importing  “logic”  or  instructional  strategy 
objects,  as  well  as  “content”  objects,  the  Server  may  also  acquire  expert  tutoring 
capabilities. 

This  vision  and  view  of  instruction  and  performance  aiding  conducted  through 
human-computer  conversations  seem  inevitable.  Sooner  or  later,  computer  and  instruc¬ 
tional  technology  will  produce  personal  learning  associates  that  have  the  functionalities 
envisioned  by  ADL.  The  ADL  initiative  seeks  to  bring  about  instructional  efficiency  and 
knowledge  accessibility  sooner  rather  than  later.  Although  difficult  issues  remain  to  be 
resolved,  the  most  intransigent  barriers  to  this  future  may  be  those  arising  from  the  cur¬ 
rent  institutions,  organizational  structures,  instructional  practices,  and  administrative 
policies  that  are  now  vested  in  instruction  conducted  as  lessons  for  students  gathered 
together  in  one  place  and  at  one  time  (i.e.,  instruction  conducted  in  classrooms  and  not  as 
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tutorial  conversations  available  to  individuals  anytime,  anywhere).  Although  ADL  tech¬ 
nologies  can  be  used  as  easily  in  the  classroom  as  not,  their  impact  will  be  to  shift  the 
emphasis  in  instructional  practice,  organizations,  and  approaches  away  from  classrooms 
and  onto  individual  users  (Fletcher  and  Tobias,  2003). 

D.  WHAT  ABOUT  INSTRUCTIONAL  DESIGN? 

With  SCORM,  ADL  has  begun  a  process  to  populate  the  cloud  on  the  left  side  of 
Figure  1.  The  process  assumes  continued  development  of  sharable  instructional  objects; 
continued  development,  expansion,  and  use  of  the  World  Wide  Web;  continued  influence 
of  Moore’s  Law  on  the  cost  and  capabilities  of  electronics;  and  increasingly  powerful, 
portable,  and  affordable  electronic  devices,  which  are  depicted  on  the  right  side  of  Fig¬ 
ure  1.  How  might  the  Server  (in  the  middle  of  Figure  1)  operate  to  assemble  instructional 
objects  into  effective  instructional  (and  performance-aiding)  interactions?  How  might  it 
design  these  interactions? 

At  present,  a  gap  exists  between  the  one-of-a-kind,  monolithic  products  of  CBI 
development  and  ITSs  and  the  object-based  development  targeted  by  ADL.  The  SCORM 
specifications  are  intended  to  make  instructional  components  sharable  and  reusable  by 
either  computers  and/or  the  human  designers  who  assemble  these  components  into 
instructional  (and  performance-aiding)  interactions.  These  specifications  attempt  to 
bridge  the  gap  between  ADL  goals  and  current  instructional  design  by  providing  a  foun¬ 
dation  for  object-based  assembly  that  begins  with  relatively  small,  reusable  learning 
resources,  which  are  then  aggregated  to  fonn  units  of  instruction  (or  performance  aiding). 
By  themselves,  the  objects  may  have  no  specific  context.  However,  when  they  are  com¬ 
bined  with  other  objects,  the  resulting  aggregation  provides  context  and  enables  interac¬ 
tions  with  users  in  a  sustained  instructional  or  problem-solving  conversation. 

Working  forward  from  SCORM,  we  might  well  ask  how  its  specifications  and 
operating  principles  for  instructional  objects  affect  and  shape  the  process  of  instructional 
design.  SCORM  has  evolved  through  a  series  of  versions,  each  intended  to  build  on — 
rather  than  replace — previous  versions.  Its  organizational  structure  may  be  described  as  a 
set  of  semi-independent  functions.  The  SCORM  specifications  (currently  available  at 
http://www.adlnet.org/)  integrate  and  harmonize  application  details  and  requirements 
drawn  from  various  standards  and  other  specifications.  They  can  be  summarized  as  the 
following: 
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•  The  SCORM  Content  Aggregation  Model  describes  how  to  develop  shar- 
able  instructional  objects,  package  them  for  exchange  from  system  to  system, 
and  describe  them  for  search  and  discovery. 

•  The  SCORM  Run-Time  Environment  describes  the  Learning  Management 
System  (LMS)  (or  Server)  requirements  needed  to  manage  the  ADL  run-time 
environment.  It  covers  materials  launch,  communication  between  materials 
and  Server,  data  model  elements  for  sharing  information  about  the  learner’s 
progress  and  needs,  and  so  forth. 

•  The  SCORM  Sequencing  and  Navigation  describes  how  SCORM  materials 
can  be  sequenced  through  a  set  of  learner-initiated  or  system-initiated 
navigation  events.  Branching  and  flow  can  be  described  by  a  predefined  set 
of  activities,  determined  at  design  time  or  generated  as  needed  on-demand. 

Although  the  SCORM  Content  Aggregation  and  Run-Time  Environment  specifi¬ 
cations  present  functions  essential  to  ADL,  the  SCORM  Sequencing  and  Navigation  is 
the  specification  that  most  profoundly  affects  the  process  of  instructional  design.  With 
“sequencing,”  software  engineers  discuss  what  instructional  designers  might  call 
“branching,”  and,  with  “navigation,”  they  address  issues  that  instructional  designers 
might  call  “learner  control.”  The  SCORM  specifications  ensure  that  the  sequencing  and 
navigation  established  by  any  confonnant  development  environment  using  any  confor¬ 
mant  tools  will  operate  successfully  and  as  required  by  the  developer(s)  in  any  SCORM- 
confonnant  run-time  environment. 

The  SCORM  Sequencing  and  Navigation  specifications  are  based  on  “use  cases” 
drawn  from  already  delivered  and  currently  used  technology-based  instruction.  These 
examples  and  the  Sequencing  and  Navigation  specifications  developed  for  them  key  on 
techniques  that  are,  at  base,  what  instructional  designers  may  recognize  as  Keller’s  Per¬ 
sonalized  System  of  Instruction  (PSI)  (Keller,  1968)  and  Crowder’s  intrinsic  program¬ 
ming  (Crowder,  1959). 

Keller’s  PSI  involves  a  process  of  breaking  up  a  course  of  instruction  into  an 
ordered  series  of  modules  and  then  pre-testing  students  for  their  mastery  of  each 
module’s  content  before  beginning  work  in  it  (Keller,  1968).  Students  who  pass  the  pre¬ 
test  skip  the  module  and  proceed  to  a  pre-test  for  the  next  module  in  the  series.  Students 
who  do  not  pass  the  pre-test  are  required  to  complete  the  module  and  then  be  retested. 
They  repeat  this  process  until  they  pass  the  test — in  practice,  one  of  several  parallel 
tests — and  only  then  do  they  proceed  to  the  next  module. 
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Various  studies  have  found  PSI  to  be  effective.  A  meta-analysis  of  75  empirical 
comparisons  of  PSI  with  standard  classroom  practices  was  reported  by  Kulik,  Kulik,  and 
Cohen  (1979).  They  found  that  the  PSI  programs  they  reviewed  raised  final  examination 
scores  by  about  0.50  standard  deviations,  roughly  an  increase  in  the  performance  of  50th 
percentile  students  to  that  of  69th  percentile  students.  They  also  found  that  PSI  produced 
less  variation  in  achievement,  higher  student  ratings,  and  fewer  course  withdrawals. 
Despite  these  favorable  results,  Keller  (1985)  grew  pessimistic  about  the  use  of  PSI 
because  of  the  substantial  amount  of  instructor  time  required  to  set  up  PSI  courses  and 
the  lack  of  support  (mostly  in  the  form  of  release  time)  from  administrators. 

Crowder’s  intrinsic  programming  (1959)  is  interesting  because  it  and  not  Skin¬ 
ner’s  approach  (1954)  is  the  one  almost  exclusively  found  in  practice  even  though  Skin¬ 
ner’s  extrinsic  programming  is  most  frequently  cited  as  the  backbone  of  programmed 
instruction.  Crowder’s  approach  will  be  familiar  to  most  instructional  developers.  An 
example  is  the  following: 

In  the  multiplication  3x4  =  12,  the  number  12  is  called  a _ . 


A: 

Factor 

{Branch  to  remedial  XI } 

B: 

Quotient 

{ Branch  to  remedial  X2J 

C: 

Product 

{ Reinforce ,  go  to  the  next  item} 

D: 

Power 

{ Branch  to  remedial  X3}. 

In  this  item,  the  system,  the  computer  instructor  assumes  that  a  student  choosing 
“A”  misunderstands  the  meaning  of  “Factor,”  or  lacks  an  understanding  of  “Product,”  or 
both.  The  student  is  branched  to  instructional  items  intended  to  correct  one  and/or  the 
other  of  these  cognitive  states  and  is  then  returned  to  this  or  a  similar  item  to  try  again.  A 
similar  remedial  approach  is  applied  if  answers  “B”  and  “D”  are  chosen.  A  student  who 
chooses  “C”  is  usually  rewarded  (“reinforced”)  with  encouraging,  or  positive,  feedback 
and  then  sent  on  to  whatever  item  best  continues  his  or  her  progress  through  the  instruc¬ 
tion,  an  action  that  by  itself  may  constitute  positive  reinforcement. 

These  two  approaches  for  sequencing  between  modules  (PSI)  and  within  modules 
(intrinsic  programming)  provide  the  bases  for  much  of  the  CBI  delivered  today  and, 
either  consciously  or  not,  the  use  cases  that  the  SCORM  Sequencing  and  Navigation 
specifications  were  intended  to  accommodate.  Figure  2  shows  a  comparison  between  CBI 
branching  and  SCORM  sequencing.  The  typical  CBI  branching  structure  on  the  left 
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Figure  2.  Comparison  of  CBI  Branching  and  SCORM  Navigation 

would  be  implemented  as  shown  on  the  right  by  using  SCORM  Sequencing  and  Naviga¬ 
tion.  Both  sides  of  the  figure  depict  typical  Keller  PSI  branching,  although  it  is  not  diffi¬ 
cult  to  see  how  the  same  schemes  might  be  extended  to  within-module  Crowder  intrinsic 
programming.  Boxes  A,  B,  and  C  can  all  be  instructional  objects  (even  if  one  or  more  of 
them  is  an  assessment  module),  or  they  can  be  assembled  from  more  granular  objects  by 
the  Server. 

The  elements  shown  in  Figure  2  can  be  used  as  a  basis  for  an  entire  course  by 
repeating  and  extending  them  into  a  comprehensive  tree  structure  that  starts  with  a  single 
entry  point  and  branches  out  to  as  many  modules  as  are  needed.  In  this  way,  SCORM 
Sequencing  and  Navigation  accommodate  Keller’s  and  Crowder’s  approaches  and  others 
as  well.  Learners  proceed  through  the  tree  from  module  to  module  until  they  run  out  of 
modules  and  can  proceed  no  further.  At  that  point,  they  are  done  with  the  course,  unit,  or 
lesson.1 

The  SCORM  Sequencing  and  Navigation  specifications  then  assume  that  most 
learning  design  strategies  can  be  represented  as  a  tree  of  learning  modules  and  that 
instructional  designers  will  be  able  to  map  their  instructional  designs  and  strategies  into 
this  structure.  SCORM  allows  access  to  global  variables  that  can  be  used  to  communicate 
data  from  one  module  or  object  in  the  tree  to  another.  It  also  allows  modules  to  be  invisi- 


1  Figure  2  is  adapted  from  SCORM  Best  Practices  Guide  for  Content  Developers  (2003),  prepared  by 
the  Carnegie  Mellon  Learning  Systems  Architecture  Lab  (LSAL)  (http://www.lsal. emu. edu/ )  to  assist 
instructional  developers.  Readers  who  want  more  information  about  implementing  instructional 
strategies  under  SCORM  Sequencing  and  Navigation  are  encouraged  to  considt  this  publication. 
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ble  to  learners,  thereby  permitting  the  use  of  specialized  computations  to  implement  more 
elaborate  instructional  approaches,  such  as  those  involving  Bayesian  networks  (e.g., 
Conati,  Gertner,  VanLehn,  and  Druzdzel,  1997;  Corbett,  Koedinger,  and  Anderson, 
1997).  These  approaches  should  be  able  to  function  in  the  SCORM  tree  structure,  deter¬ 
mining  if  users  are  on  the  right  track,  shaping  the  scope  and  sequence  of  instruction  they 
receive,  and  guiding  their  progress.  Hidden  Markov  Models,  such  as  those  used  by  Soller 
and  Lesgold  (2003)  to  facilitate  collaboration  among  physically  distributed  participants, 
can  also  be  incorporated  in  a  similar  fashion  to  help  participants  collaborate  in  solving 
problems. 

SCORM  Sequencing  and  Navigation  specifications  are  new  and  await  empirical 
validation  to  determine  the  extent  to  which  they  support  these  and  other  approaches.  The 
question  naturally  arises  as  to  how  much  farther  SCORM  Sequencing  and  Navigation  can 
be  pushed  toward  the  ADL  vision  in  which  instruction  consists  less  of  predefined  lessons, 
testing,  and  screening  and  becomes  more  of  a  conversation  between  the  learner/user  and 
the  technology. 

Empirical  demonstrations  to  answer  this  question  can  be  showcased  through 
“Designfests,”  which  are  analogs  of  the  current  ADL/SCORM  “Plugfests.”  The  ADL  ini¬ 
tiative  uses  Plugfests  to  assess  and  demonstrate  how  well  servers,  objects,  and  authoring 
tools  conform  to  SCORM  specifications  and,  in  turn,  how  satisfactory  the  specifications 
are  in  meeting  users’  needs.  In  a  similar  fashion,  Designfests  will  demonstrate  how  and  to 
what  extent  SCORM  specifications  can  be  used  to  implement  the  instructional  designs 
intentions  of  developers.  Designfests  will  help  identify  gaps  between  what  developers 
want  to  do  and  what  is  practicable  under  SCORM  Sequencing  and  Navigation  specifica¬ 
tion.  They  will  also  set  priorities  for  further  development  of  SCORM  specifications. 

E.  TOWARD  MORE  ADAPTIVE,  INTELLIGENT  LEARNING  SYSTEMS 

Mainstream  learning  systems  often  rely  on  predetennined  and  fixed-path  delivery 
of  content.  Such  systems  lack  agility  in  adapting  to  learners’  mastery  states  and  are 
thereby  limited  in  their  ability  to  tailor  learning  experiences  to  individual  learners.  As 
specified  some  time  ago  (e.g.,  Fletcher,  1975),  an  adaptive,  “intelligent”  learning  system 
needs  an  accurate  model  of  the  learner,  a  model  of  the  knowledge  domain,  and  a  capa¬ 
bility  that  can  evaluate  the  differences  between  the  two.  It  can  then  identify  and/or 
devise,  on-demand  and  in  real  time,  instructional  strategies  that  produce  desired 
instructional  outcomes. 
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SCORM  provides  globally  accessible  records  that  can  store  the  learner’s  degree 
of  mastery.  Included  in  the  records  are  data  elements  that  pennit  them  to  reference 
externally  defined  competencies.  As  the  learner  is  sequenced  through  the  content  objects, 
the  learning  system  builds  up  a  representation  of  the  learner’s  mastery  and  progress. 
Records  of  this  sort  comprise  a  simple,  accumulative  model  of  the  learner’s  state. 

IMS  Global  Learning  Consortium,  Inc.’s  (2002)  IMS  Reusable  Definition  of  Com¬ 
petency  or  Educational  Objective  adds  to  this  capability  by  defining  a  taxonomy  of  com¬ 
petencies  required  to  meet  specific  learning  objectives.  This  taxonomy  can  be  organized 
hierarchically  to  represent  dependencies,  supporting  skills,  or  prerequisites.  Each  compe¬ 
tency  definition  includes  a  text  description  of  the  competency  and  a  unique  identifier  that 
can  be  referenced  externally.  The  organization  of  a  competency  definition  may  represent 
specific  skills  or  knowledge  to  be  acquired  for  a  specific  task  or  subject  domain.  By 
referencing  Competency  Model  identifiers,  SCORM  records  can  be  used  to  compare  the 
state  of  the  learner  with  the  generic  IMS  Global  Learning  Consortium,  Inc.  competencies. 
This  capability  provides  a  generalizable,  system-based  means  to  perform  knowledge-  and 
skills-gap  analyses,  which  lead  to  the  more  sophisticated  and  adaptive  strategies  that  use 
such  information  (Wiley,  2000). 

As  learning  system  specifications  become  more  robust,  they  will  also  become 
more  adaptive.  Improved  assessment  methods  and  results  that  will  continuously  and 
unobtrusively  extract  information  from  instructional  interactions  and  better  represent  the 
state  of  the  learner  are  emerging.  The  strategies  developed  by  learning  systems  will  be 
informed  further  by  learner  profile  information,  which  can  “pre-load”  the  learner  model 
with  mastery  infonnation  from  outside  sources,  thereby  reducing  the  need  for  additional 
testing  to  determine  the  learner’s  state.  This  process  improves  the  capabilities  of  tech¬ 
nology-based  instructional  systems  to  bypass  the  relevant  content  of  already-mastered 
material  and  concentrate  on  relevant  material  yet  to  be  learned — a  process  that  has  long 
been  advocated  by  researchers  (e.g.,  Tobias,  1989). 

Basically,  we  seek  an  engineering  of  instruction  with  well-articulated  principles 
for  adjusting  and  modulating  learning  experiences.  Such  engineering  would  ensure  that 
outcomes  such  as  retention  of  skills  and  knowledge,  application  and  transfer  of  learning, 
motivation  to  continue  study,  speed  of  response,  accuracy  of  response,  and  so  forth  are 
achieved  reliably  by  each  learner  to  the  maximum  extent  possible  within  the  constraints 
imposed  by  instructional  time  and  resources.  This  instructional  engineering  would  auto¬ 
matically  locate  precisely  appropriate  objects  and  assemble  them  as  needed  for  interac- 
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tions  with  the  learner.  Each  interaction  would  be  tailored — on-demand  and  in  real  time — 
to  the  outcome  being  sought,  to  the  learner’s  level  of  knowledge,  skill,  and  style  of 
learning,  and  to  the  instructional  strategy  indicated  by  instructional  principles.  This  is  a 
significant  challenge  for  instructional  objects,  Web-based  services,  and  the  state  of  the  art 
in  general,  but  current  progress  suggests  that  technology  may  eventually  rise  to  meet  this 
challenge,  yielding  instruction  that  ensures  reliable  achievement  of  targeted  instructional 
outcomes  by  all  students. 

1.  How  Might  the  Web  and  Web  Services  Impact  This  Evolution? 

One  way  the  current  and  near-tenn  capabilities  of  learning  systems  can  evolve  is 
through  the  Semantic  Web,  which  will  provide  powerful  new  technologies  for  both 
knowledge  representation  and  the  ontologies  needed  to  connect  them  (Berners-Lee, 
Hendler,  and  Lassila,  2001).  These  technologies  will  provide  ways  not  only  to  relate,  but 
also  to  reason  about  infonnation  from  widely  different  domains. 

The  Semantic  Web  is  intended  to  imbue  information  available  on  the  Web  with 
sufficient  meaning  to  improve  significantly  the  cooperation  between  computers  and 
human  beings.  Dealing  with  the  semantic  content  of  Web  pages  and  information  will 
improve  the  process  of  discovery  needed  to  access  relevant  information  and  objects  from 
the  Web.  Access  to  this  semantic  content  will  key  on  the  development,  implementation, 
and  use  of  ontologies,  which  make  it  possible  to  identify  and  expose  semantic  linkages 
between  highly  disparate  bodies  of  information  (Chandrasekaran,  Josephson,  and 
Benjamins,  1999). 

Roughly,  an  ontology  consists  of  a  taxonomy  and  a  set  of  inference  rules  that  for¬ 
mally  define  operations  and  relations  among  the  classes  defined  by  the  taxonomy.  More 
specifically,  ontologies  consist  of  consensual,  shared,  fonnal  descriptions  that  identify 
classes  of  objects,  each  member  of  which  possesses  all  qualities  that  all  other  members  of 
the  class  have  in  common.  The  classes  are  organized  in  hierarchies,  and  classes  of  classes 
can  be  developed  to  any  necessary  depth.  Relationships  between  a  member  of  any  class 
defined  by  an  ontology  can  not  only  be  quickly  linked  to  many  other  classes  and  class 
members,  but  the  semantic  quality  that  forms  the  link  can  also  be  exposed.  Ontologies 
identify  semantic  links  between  what  may  appear  to  be  quite  disparate  classes  and  class 
members.  Web  services  that  identify  and  exploit  these  semantic  linkages,  and,  in  general, 
increase  the  “behavioral  intelligence”  of  Web-based  applications  are  being  devised  and 
implemented — as  Bryson  et  al.  (2002)  have  suggested. 
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These  Web  services  are  being  built  on  top  of  existing  and  emerging  Web  stan¬ 
dards,  such  as  Hyper-Text  Transfer  Protocol  (HTTP),  Extensible  Markup  Language 
(XML),  Universal  Description,  Discovery,  and  Integration  (UDDI),  and  Simple  Object 
Access  Protocol  (SOAP).  In  this  way,  emerging  services  are  being  made  language,  plat¬ 
form,  and  object  model  independent.  They  enable  different  applications  running  on 
different  operating  systems  developed  with  different  object  models  using  different  pro¬ 
gramming  languages  and  programming  environments  to  cooperate,  communicate,  and 
interoperate.  They  can  express  complex  relationships  using  inference  rules  (e.g.,  those  of 
ITSs)  to  perform  specific  tasks  such  as  profiling  learners,  representing  their  skills,  knowl¬ 
edge,  and  abilities,  linking  these  representations  to  instructional  objects,  and  managing 
their  progress  toward  instructional  objectives  and  competencies. 

If  successful,  the  Semantic  Web  will  integrate  real-world  knowledge  and  skills 
acquired  through  simulation,  education,  training,  performance  aiding,  and  experience.  It 
will  provide  a  foundation  for  building  more  comprehensive  and  substantive  models  of 
subject  matter  domains  and  learners’  levels  of  mastery  than  we  now  have  and  combine 
them  with  a  more  precise  discovery  of  the  instructional  objects  that  learners  and  other 
users  need  in  order  to  develop  desired  human  competencies.  Building  on  the  already 
available  functionalities  of  ITSs,  sharable  objects,  and  existing  standards,  the  Semantic 
Web  and  its  services  will  contribute  substantially  to  the  next  generation  of  learning 
environments. 

2.  Content  Object  Discovery  and  Retrieval 

Given  these  considerations,  it  is  not  surprising  that  the  development  of  Web  ser¬ 
vices  used  to  identify  and  retrieve  contextually  relevant  instructional  content  is  becoming 
a  major  topic.  The  success  of  Google  and  other  Web  search  engines  has  demonstrated  the 
value  and  utility  of  content  discovery  and  has  whetted  everyone’s  appetite  for  rapid, 
accurate  search  and  retrieval.  Presently,  Google  may  be  the  single  most  important, 
effective,  and  widely  used  source  of  Web-based  education.  However,  Google’s  location 
of  content  by  text  crawling,  indexing,  and  retrieving  everything  that  is  remotely  relevant 
to  a  search  has  limited  its  use  as  a  discovery  system  for  focused  content  assembly.  Its 
operation  could  be  improved  substantially  if  it  would  cooperate  with  content  and  retrieve 
only  what  is  intentionally  prepared  and  published  for  discovery. 

More  precise  identification  of  content  objects  is  being  addressed  through  the  use 
of  Uniform  Resource  Names  (URNs)  (http://www.faqs.org/rfcs/rfcl737.html),  which 
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serve  as  persistent,  location-independent  resource  identifiers.  The  Corporation  for 
National  Research  Initiatives  (CNRI)  has  created  a  URN  implementation  called  The  Han¬ 
dle  System  (Kahn  and  Wilensky,  1995;  http://www.handle.net/introduction.html).  This 
system  allows  digital  objects  to  obtain  a  unique  identifier  and  link  each  object  to  its  loca¬ 
tion — wherever  that  might  be — through  the  use  of  a  Handle  Resolution  Service  (similar 
to  domain  names  resolving  to  Internet  protocol  addresses  through  the  Domain  Name 
System).  CNRI  hosts  a  global  root  server  that  can  be  queried  to  resolve  requests. 

Also,  the  Common  Indexing  Protocol  (CIP)  (http://www.faqs.org/rfcs/ 
rfc265 1  .html)  allows  the  owner  of  content  to  create  its  index  metadata  while  also 
allowing  this  indexing  information  to  be  shared  among  different  servers,  thereby  enabling 
the  development  of  new  search  and  discovery  services.  New  learning  and  performance- 
aiding  specifications  are  emerging,  and  these  specifications  permit  the  identification  of 
skills,  competencies,  and  knowledge  so  that  logical  relations  among  them  that  are  rele¬ 
vant  to  specific,  but  quite  different  communities  of  practices  can  be  identified  and  then 
represented.  As  suggested  earlier,  not  only  will  such  logical  relations  be  discovered,  but 
the  semantic  nature  of  these  relationships,  insofar  as  they  are  reflected  in  metadata  defini¬ 
tions,  will  be  exposed. 

These  developments,  among  others,  may  produce  Web  services  that  provide  accu¬ 
rate,  precisely  focused,  and  contextually  correct  discovery  and  retrieval  of  instructional 
objects  on  an  easily  scalable  basis.  Their  combination  of  agility  and  accuracy  enables 
considerable  flexibility  in  dealing  with  the  idiosyncratic  prior  knowledge  elements  and 
associations  built  up  by  individual  users.  They  will  allow  instructional  programs  to 
assemble  continuously  and  unobtrusively  models  of  each  user’s  state  of  knowledge,  style 
of  learning,  and  progress  toward  instructional  objectives.  These  models,  in  turn,  will  sup¬ 
port  the  precise  tailoring  of  instructional  interactions  to  each  student,  which  is  a  charac¬ 
teristic  and  unique  strength  of  one-on-one  tutoring.  They  will  provide  an  Aristotle  for 
every  Alexander  and  a  Mark  Hopkins  for  the  rest  of  us. 

F.  WHERE  MIGHT  THESE  CAPABILITIES  TAKE  US? 

The  emphasis  on  instructional  technology  brings  us  to  revolutions  in  instruction. 
The  first  of  these  may  have  occurred  with  the  development  of  written  language  about 
7,000  years  ago.  It  allowed  the  content  of  advanced  ideas  and  teaching  to  transcend  time 
and  place.  The  second  revolution  in  instruction  began  with  the  technology  of  books. 
Books  made  the  content  of  high-quality  instruction  available  anywhere  and  anytime  and 
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were  also  inexpensive  way  to  reach  many  more  people.  A  third  revolution  in  instruction 
appears  to  be  the  introduction  of  computer  technology.  The  capability  of  this  technology 
for  real-time  adjustment  of  instructional  content,  sequence,  scope,  difficulty,  and  style  to 
meet  the  needs  of  individuals  suggests  a  pervasive  and  significant  revolution  in  instruc¬ 
tion.  It  makes  the  content  and  the  interactions  of  high-quality  instruction  widely  and 
inexpensively  accessible — again  anytime,  anywhere. 

Building  on  this  possibility,  ADL,  SCORM,  intelligent  tutoring,  and  the  Semantic 
Web  in  some  combination  may  provide  a  foundation  for  generative  education,  training, 
and  performance-aiding  capabilities  that  are  available  anytime,  anywhere.  These 
developments  can  capitalize  on  the  growth  of  electronic  commerce  and  the  World  Wide 
Web.  They  can  build  on  this  worldwide,  almost  irresistible  activity,  accelerate  it,  and 
apply  it  to  a  full  spectrum  of  education,  training,  and  perfonnance-aiding  needs. 
However,  to  realize  all  this,  we  must  also  leam  to  combine  the  software  engineering 
features  offered  by  SCORM  with  the  best  we  have  to  offer  in  the  form  of  instructional 
design. 

The  long-term  anytime,  anywhere  vision  for  ADL  differs  substantially  from  class¬ 
room  learning  and  the  many  organizational  structures  we  have  in  place  to  support  it,  but 
ADL  is  not  at  odds  with  classroom  practice.  Anytime,  anywhere  includes  classrooms,  and 
ADL  capabilities  are  as  accessible  in  classrooms  as  they  are  elsewhere.  The  instructional 
and  perfonnance-aiding,  human-computer  conversations  that  are  the  eventual  goal  of 
ADL  will  access  the  comprehensive  spectrum  of  human  knowledge  becoming  available 
from  the  World  Wide  Web  and  tailor  it  to  the  user’s  needs.  These  conversations  will  ini¬ 
tially  be  designed  to  mimic  those  that  are  established  by  human  tutors,  but  sooner  or  later 
this  guiding  metaphor  must  evolve  and  these  conversations  will  take  on  forms,  capabili¬ 
ties,  and  infrastructure  of  their  own.  The  “Columbus  Effect”2  will  take  over  just  as  it  did 
for  wireless  telegraph,  horseless  carriages,  and  a  host  of  other  technological  innovations 
that  led  us  into  territory  not  envisioned  in  the  original  enabling  metaphor. 


2  Prognostications  aside,  technological  approaches  to  instruction  may  provide  yet  another  example  of 
what  might  be  called  the  Columbus  Effect.  As  readers  will  recall,  Columbus  sailed  west  intending  to 
find  India  (and  a  lucrative  spice  route).  Instead,  he  (re-)  discovered  what  became  a  new  world  for 
Europeans.  Such  a  result  typifies  technological  progress.  Seeking  one  thing  based  on  familiar, 
common  practice,  we  inevitably  end  up  with  something  else — unforeseen  and  unexpected.  Wireless 
telegraph  produced  something  functionally  quite  different  from  the  telegraph — namely,  the  radio. 
Similarly,  efforts  to  make  a  carriage  move  without  a  horse  produced  automobiles,  to  say  nothing  of  gas 
stations,  motels,  and  the  Santa  Monica  Freeway.  In  seeking  affordable,  one-on-one  tutoring  through 
automation,  we  may  end  up  with  something  no  one  now  envisions. 
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At  least  three  capabilities  may  evolve  from  the  ADL  teaching-learning 
environment: 

1.  Less  predefined  sequencing.  An  instruction  (or  perfonnance-aiding)  con¬ 
versation  will  presumably  take  whatever  direction  is  needed  by  participants 
in  the  conversation.  Develop  and  providing  a  capability  that  allows 
sequences  to  adjust  and  evolve  continually — perhaps  a  meta-sequencing 
capability — is  a  significant  challenge  for  instructional  designers.  The  notion 
of  instructional  design  as  a  process  of  prespecifying  and  predefining  a 
sequence  of  activities  within  a  lesson  module  will  need  to  evolve 
substantially  if  the  ADL  vision  of  a  conversation  sustained  on  an  interaction- 
to-interaction  basis  is  to  be  realized  fully. 

2.  More  assessment  and  fewer  tests.  Assessment  will  become  more  continu¬ 
ous  and  unobtrusive  as  the  capability  for  developing  a  model  of  the  learner/ 
user  from  interactions  evolves.  Such  assessment  can  be  accomplished  by 
taking  account  of  the  learner’s  vocabulary,  use  of  technical  infonnation,  level 
of  abstraction,  clustering  (chunking)  of  concepts,  inferred  hypothesis  forma¬ 
tion,  and  so  forth.  These  capabilities  have  not  yet  been  explored  and  verified 
fully,  but  enough  research  on  their  application  has  been  completed  to  suggest 
their  promise  for  the  continuous  and  unobtrusive  assessment  of  user  knowl¬ 
edge  and  abilities  needed  to  tailor  instruction  and  performance  aiding  to  their 
needs.  Some  explicit  testing  and  explicit  probing  may  still  be  used  to  assess 
learner  progress  efficiently.  What  sort  of  probes  are  needed,  how  they  are  to 
be  implemented,  and  what  principles  will  guide  their  psychometric  properties 
are  other  challenges  for  instructional  designers — challenges  that  should  not 
be  left  to  evaluators  and  the  testing  community  as  a  separate,  “stove-piped” 
activity,  but  one  that  also  integrates  evaluation  and  instructional  design. 

3.  Fewer  lessons  and  more  learning.  The  notion  of  monolithic  instructional 
modules  intended  to  achieve  instructional  objectives  will  also  need  to  evolve 
if  instruction  and  perfonnance-aiding  conversations  are  to  be  supported. 
Objectives  may  need  to  be  specified  more  finely  by  more  comprehensive 
hierarchical  decomposition  than  those  called  for  by  current  instructional 
design.  As  suggested,  a  capability  is  needed  to  treat  instruction  not  as  art  or 
science,  but  as  engineering,  where  specific  outcomes,  based  on  the  detailed 
knowledge  of  the  learner/user  matched  with  comprehensive  representations 
of  the  subject  matter,  can  be  achieved  reliably  by  all  learners — even  when  the 
targeted  outcomes  themselves  are  modified  on  the  fly. 

Other  challenges  may  also  occur  to  the  reader.  Issues  of  privacy  and  security, 
integration  with  current  instructional  practices,  certification  at  a  distance,  and  the  balance 
between  individual  learning  and  the  need  for  social  interaction  remain  as  topics  for 
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research,  development,  and  implementation,  but  they  do  not  seem  as  peculiar  to  the  ADL 
vision  for  distributed  learning  as  the  three  capabilities  listed  previously  (see  p.  19). 


G.  ARE  THESE  LEARNING  ENVIRONMENTS  WORTHWHILE? 

Hundreds  of  evaluations  have  been  performed  to  assess  the  interactive  instruc¬ 
tional  capabilities  incorporated  in  ADL.  As  reviewed  in  more  detail  by  Fletcher  (2003), 
the  case  for  using  these  technology-mediated  learning  environments  (based  on  empirical 
data)  may  be  roughly  summarized  as  follows: 

•  Tailoring  instruction  (education  and  training)  to  the  needs  of  individual  stu¬ 
dents  has  been  found  to  be  an  instructional  imperative  and  an  economic 
impossibility. 

•  Technology  can,  in  many  cases,  make  this  instructional  imperative  afford¬ 
able.  Under  any  appreciable  student  load,  providing  instruction  with  digital 
technology  is  less  expensive  than  hiring  a  tutor  for  each  student. 

•  Technology-based  instruction  has  been  found  to  be  more  effective  than  cur¬ 
rent  classroom  instructional  approaches  in  many  settings  across  many  subject 
matters. 

•  Technology-based  instruction  is  generally  less  costly  than  current  instruc¬ 
tional  approaches,  especially  when  many  students  are  to  be  trained  or  when 
instructional  objectives  concern  operating  or  maintaining  costly  equipment. 

•  Technology-based  instruction  has  been  found  to  decrease  the  time  needed  to 
reach  targeted  instructional  objectives. 

Overall,  a  rule  of  “thirds”  emerges  from  CBI  assessments.  Findings  suggest  that 
use  of  interactive  instructional  technologies  reduces  the  cost  of  instruction  by  about 
one-third,  and  it  either  reduces  time  of  instruction  by  about  one -third  or  it  increases  the 
amount  of  skills  and  knowledge  acquired  by  about  one-third.  Similar  if  not  enhanced 
results  can  reasonably  be  expected  with  the  instructional  capabilities  that  ADL  adds  to 
basic  technology-based  instruction.  These  results,  combined  with  anywhere,  anytime 
accessibility,  also  provided  by  technology-based  instruction,  suggest  the  value  of 
achieving  the  ADL  vision. 

H.  DISTANCE  LEARNING  AND  ADL 

ADL  approaches  contrast  with  less  interactive,  less  agile,  and  less  flexible  tech¬ 
nologies  such  as  video  teletraining,  video  conferencing,  instructional  radio,  paper-based 
correspondence  instruction,  and  instructional  telephone,  all  of  which  have  been  used  to 
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provide  distance  learning.  In  general,  distance-learning  studies  using  these  less 
interactive  technologies  find  that  they  provide  instruction  that  is  about  as  effective  as 
residential  classroom  instruction  and  is  less  preferred  by  students  but  is  notably  less 
costly.  For  instance,  Russell  (1999)  identified  355  studies  reporting  no  significant 
differences  between  distance  education  and  other  instructional  approaches.  His  findings 
are  confirmed  by  other  researchers  (Lockee,  Burton,  and  Cross,  1999;  Phipps  and 
Merisotis,  1999;  Bernard,  Lou,  and  Abrami.  2003). 

Bernard  et  al.  (2004)  reported  a  meta-analytic  review  of  232  studies  comparing 
distance  education  with  classroom  instruction.  Among  other  things,  they  found  superior 
results  for  classroom  instruction  when  synchronous  approaches  to  distance  education 
were  used  and  superior  results  for  distance  education  when  asynchronous  approaches 
were  used.  In  a  meta-analytic  review  of  105  empirical  studies  comparing  Web-based 
learning  with  classroom  approaches,  Sitzmann  et  al.  (2004)  found  Web-based  instruction 
to  be  more  effective  for  teaching  declarative  knowledge,  but  they  found  virtually  no  dif¬ 
ference  in  the  effectiveness  of  the  two  forms  of  instruction  for  teaching  procedural 
knowledge.  In  any  case,  lower  costs  and  improved  accessibility  suggest  superior  cost- 
effectiveness  for  distance  education  even  when  research  finds  it  to  be  no  more  than 
equally  as  effective  as  classroom  instruction. 

I.  FINAL  WORD 

Much  remains  to  be  done.  The  vision  or  view  of  the  future  presented  in  this 
document  is  not  likely  to  be  accomplished  soon,  but  it  seems  likely  to  be  accomplished 
because  of  our  progress  in  such  areas  as  electronics,  computer  technology,  computer 
communications,  and  knowledge  representation.  Serious  issues  remain  in  the 
development  of  instructional  strategies  that  reliably  lead  from  the  learner’s  (or  user’s) 
present  state  of  knowledge,  skill,  and  perfonnance  to  one  that  is  targeted  and  desired.  We 
need  a  capability  that  is  neither  art  nor  science  but,  instead,  is  most  analogous  to 
engineering  where  known  principles  are  applied  to  achieve  specified  outcomes  in,  if  we 
are  fortunate,  a  cost-effective  manner. 

ADLs  anytime,  anywhere  objectives  are  not  contrary  to  classroom  instruction  but 
are  very  different.  They  will  require  changes  in  the  roles  and  responsibilities  of  students, 
instructors,  and  administrators.  The  budgeting  practices  and  organizational  structures 
now  focused  on  classroom  settings  will  also  require  major  modifications.  Like  all 
changes,  these  are  likely  to  be  painful  and,  most  certainly,  difficult  to  achieve.  The  prize, 


21 


however,  may  be  worth  the  effort.  Enabling  the  totality  of  human  knowledge  to  be 
affordable  and  available  to  every  individual  who  seeks  it  seems  to  be  a  worthy  goal. 
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